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LOW COMPLEXITY HIGH PERFORMANCE DECODER 
And METHOD of DECODING For 
COMMUNICATIONS SYSTEMS USING MULTIDIMENSIONAL SIGNALING 

5 Claim To Priority Of Provisional Application 

This application claims priority under 35 U.S.C. § 119(e)(1) of provisional 
application serial number 60/404,858, docket number TI-35081PS, entitled Two-Layer 
Interference Cancellation for Communications Systems Using Multidimensional 
Signaling, filed 08/21/02, by David J. Love, Srinath Hosur and Anuj Batra; and 
10 provisional application serial number 60/405,089, docket number TI-35083PS, entitled 
Low-Complexity Bit Based Reduced Subspace Maximum Likelihood Decoding for 
Communications Systems Using Multidimensional Signaling, filed 08/21/02, by David J. 
Love, Srinath Hosur and Anuj Batra. 

15 Background of the Invention 

1. Field of the Invention 

This invention relates generally to communications systems using 
multidimensional signaling. More particularly, this invention relates to a sub-optimal 
20 minimum distance decoder and method of decoding that outperforms known sub-optimal 
decoders used in multiple-input multiple-output (MIMO) communication systems. 

2. Description of the Prior Art 

Multiple-input multiple-output (MIMO) communication systems provide gains in 
25 capacity and quality compared to single-input single-output (SISO) communication 
systems. Examples of MIMO systems include but are not limited to: 1. A narrowband 
wireless communication system employing multiple-antenna at the transmitter and/or 
receiver; 2. A broadband communication system employing orthogonal frequency 
division multiplexing (OFDM); 3. A time division multiple access system; 4. Any 
30 multiuser communication system; and/or 5. Any combination of 1-4 above. 
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These systems commonly employ a block structure where the transmitter (here 
transmitter refers to the collection of SISO transmitters) sends multidimensional symbol 
information. This multidimensional symbol could, but is not limited to, be represented 
by a vector or matrix. The multidimensional symbol might represent one or more coded 
5 or uncoded SISO data symbols. The receiver (here receiver refers to the collection of 
SISO receivers) receives one of more copies of this transmitted symbol vector. The 
performance of the entire communication system hinges on the ability of the receiver to 
find reliable estimates of the multidimensional symbol that was transmitted. 

10 Definitions 



As used herein, if Q( ) represents the symbol slicing function, it will be assumed 
20 to slice both single symbols and multi-dimensional symbol vectors. 



For MIMO systems such as, but not limited to, the ones discussed herein above, 
the received signal can be written in the form 



15 



As used herein, bolded capitol symbols, such as H, represent matrices. 

As used herein, bolded lower-case symbols, such as s, represent vectors. 

As used herein, r denotes matrix transposition. 

As used herein, * denotes the matrix conjugate transpose operation. 

As used herein, A denotes the matrix inverse operation. 

As used herein, if W is a matrix, W m denotes the mth column of W. 

As used herein, if W is a matrix, (W r ) m denotes the mth row of W. 

As used herein, if v is a vector, ||v|| , denotes the 2-norm of v. 



As used herein, I M represents the M by M identity matrix. 

As used herein, Omxn represents the M by N matrix of zeros. 

As used herein, if A and B are sets, then A\B is the set of all elements in A that are 



not in B. 



25 




0) 



30 
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where H„ is an M r by M t matrix of complex gains, s* is the M t -dimensional symbol 
vector transmitted at time k> and v is a M t -dimensional vector of additive noise. In 
narrowband wireless systems where the symbol period is much larger than the RMS 
delay spread as well as OFDM systems, each SISO channel is often modeled as a single- 
5 tap complex gain. In this case equation (1) simplifies to 

y* = Hs* + v ^2) 

where H is now an M r by M t matrix of complex numbers and Hs* is the matrix product of 
10 H and S*. 

The receiver must estimate the symbol matrix S=[si ... Sr] in order to facilitate 
reliable communication. Examples, but by no means the only examples, of 
multidimensional symbols could be space-time codes where T>\ or spatial multiplexing 
15 systems with T=\ and independent SISO modulation on each transmit stream. If v were 
not present and H were invertible, this would simply be the problem of inverting H. The 
presence of noise however, increases the difficulty of estimating S. The receiver is 
generally assumed to have some estimate of H available to it. As well, the symbol matrix 
S is assumed to be chosen from a finite set C of possible multidimensional symbols. 

20 

The optimal solution in the sense of minimizing the probability of symbol error 
has been shown to be the maximum aposterior (MAP) decoder which is equivalent in the 
case of equiprobable symbol transmissions to a maximum likelihood (ML) decoder. The 

ML decoder attempts to find S, the symbol matrix, by using the symbol matrix S that 

25 maximizes /7^S|yi,....,yrj where /?(.|yi,....,yr) is the conditional probability density 

function (pdf) of s* given yi,..., y r . In real-time communications systems, however, this 
type of decoder is overly computationally complex. We will call decoders that search 

over an entire set Fand decode to the multidimensional symbol S that minimize some 



Patent Application of David James Love, Srinath Hosur and Anuj Batra 

Page 3 of 19 



Docket No.: TI-35081 



sort of metric minimum distance (MD) decoders. Examples of MD decoders include 
MAP and ML decoding with V=C. 

Algorithms have been proposed that are computationally easier than ML decoding 
5 in order to overcome this computational hurdle. Algorithms that perform some form of 
reduced complexity decoding will be referred to herein as sub-optimal decoders. An 
example of this type of decoder is successive interference cancellation (SIC). A receiver 
using SIC decodes each symbol within the symbol vector one at a time. After a symbol is 
decoded, its approximate contribution to the received vector is subtracted in order to 
10 improve the estimate of the next symbol within the symbol vector. 

An example of an SIC receiver is the ordered iterative minimum mean squared 
error (IMMSE) receiver. With a single-tap channel, the receive signal is given by 
equation (2) above. Letting s* = [s\ s 2 ... s M r] T , the ordered IMMSE operates using the 
15 following steps, letting y*,o = y*,Do = {l,2,...,M}, and H* (0) = H . 



Regarding the above algorithm, it is important to note that denotes the ith row of 
the matrix H^ +1) (time k and iteration m+7). Another example is the zero-forcing 



LSetm^O. 



2. Compute W w = (H^H^ + phf, - -^Hf* . 

3. Let « = argmin|(w (OT)r ),|| 2 . 





6. Repeat steps 1-5 for m<M t . 

1. Set the decoded symbol vector to Sk = [Saj Jk,2 ... 7k,Mi] T . 
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decoder which decodes to the symbol s* = Q(pL~ x yk). This decoder is usually 
considered the worst performing and least complex of the sub-optimal decoders. 

Sub-optimal techniques unfortunately differ in diversity order from ML decoding 
5 (i.e. the asymptotic slope of the average probability of bit error curve). They essentially 
trade reduced complexity for reduced performance. 

In view of the foregoing, it is both advantageous and desirable to find other low 
complexity schemes that, while still being low complexity, perform closer to an ML 
10 decoder. 
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Summary of the Invention 

The present invention is directed to a sub-optimal minimum distance (MD) 
decoder that outperforms known sub-optimal decoders used in multiple-input multiple- 
5 output (MIMO) communication systems. 

According to one embodiment, a sub-optimal MD decoder creates a reduced 
search space by searching over the symbol matrices corresponding to a subset E of all 
possible bit patterns of the transmitted symbol. The bit locations of the subset can be 
10 determined using some criterion such as the weakest bit positions. Note also that even 
among the selected bit locations, the bit patterns and hence the corresponding symbols 
could be further restricted to a smaller subset. The subset E could be pre-determined or 
could be adaptively deduced. 

15 According to another embodiment, a sub-optimal decoder employs an algorithm 

using reduced subspace decoding methods with an added layer of interference 
cancellation. 
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Brief Description of the Drawings 

Other aspects and features of the present invention and many of the attendant 
advantages of the present invention will be readily appreciated as the same become better 
5 understood by reference to the following detailed description when considered in 
connection with the accompanying drawings wherein: 

Figure 1 is a block diagram illustrating a reduced search space minimum distance 
decoding algorithm according to one embodiment of the present invention; 

10 

Figure 2 is a graph comparing bit error rate performance between ordered IMMSE 
based sub-ML, ordered IMMSE, and ML decoding, for 7=1 and S created by using 16- 
QAM on each stream using the algorithm depicted in Figure 1; 

15 Figure 3 is a graph comparing bit error rate performance between unordered zero- 

forcing based sub-ML, ordered IMMSE, and ML decoding for T=l and S created by 
using 16-QAM on each stream using the algorithm depicted in Figure 1; 

Figure 4 is a block diagram illustrating a two-layer interference cancellation 
20 decoder according to one embodiment of the present invention; 

Figure 5 is a graph comparing bit error rate performance of the decoder shown in 
Figure 6 with ordered IMMSE and ML decoding using T=\ and S created by using 16- 
QAM on each stream; and 

25 

Figure 6 is a graph comparing bit error rate performance of the decoder shown in 
Figure 6 with ordered IMMSE and ML decoding for a channel having two paths using 
7=1 and S created by using 16-QAM on each stream. 

30 
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While the above-identified drawing figures set forth particular embodiments, 
other embodiments of the present invention are also contemplated, as noted in the 
discussion. In all cases, this disclosure presents illustrated embodiments of the present 
5 invention by way of representation and not limitation. Numerous other modifications and 
embodiments can be devised by those skilled in the art which fall within the scope and 
spirit of the principles of this invention. 
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Detailed Description of the Preferred Embodiments 

Figure 1 is a block diagram illustrating a reduced search space maximum 
likelihood decoder 100 according to one embodiment of the present invention. Decoder 

5 100 can be used to implement a decoder that provides performance close to MAP 

decoding with only a small complexity increase over sub-optimal decoding. A received 
signal vectors yi...y* with k a positive integer are fed into a sub-optimal decoder 102 that 
may include, but are not limited to, ordered iterative minimum mean-squared error 
decoders and zero-forcing decoders. The only requirement that needs to be imposed on 

10 the soft decoder 102 is that it provides soft bit information. This soft bit information may 
be in the form, but is not limited to, log-likelihood information. The sub-optimal decoder 
102 soft output bits, 61,62,..., 6m , where N s is the number of bits per multidimensional 
symbol are then fed into a block 104 that creates a set E of search patterns. An example, 
but not the only example, of the block could be one that returns the set E of all possible 

15 bit combinations in the bit positions i\,i2,... 9 u where the positions correspond to the 
smallest magnitude soft bit locations with L being some integer. This set of search 
patterns E is next fed into a reduced search space creation unit 106. The reduced search 
space creation unit 106 creates a search space V t V e C , of multidimensional symbols. 
Each symbol vector in the search space corresponds to a different bit error pattern. 

20 Minimum distance decoding is then performed as shown in block 108 using the reduced 

search space Vto return either an estimated transmitted symbol S , soft bit information, 
or hard bit information using a metric m. Examples of the metric could be, but are not 
limited to, performing ML decoding or performing MAP decoding. 

25 Although sub-optimal minimum distance decoder 100 requires additional 

complexity compared to a simple sub-optimal decoder, decoder 100 provides numerous 
advantages over a simple sub-optimal decoder. Decoder 100 allows an adjustable search 
space size by varying the number of bit search patterns in the set E 9 as stated herein 
before. This feature allows tradeoffs between performance and complexity. Decoder 100 
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provides performance that lies close to optimal MAP decoding. Further, decoder 100 can 
be implemented as an additional stage to known existing sub-optimal decoders. 

Any type of sub-optimal decoder that outputs soft bit information can be 
5 combined with reduced search space maximum likelihood decoding, as stated herein 
before. Decoder 100 was simulated by the present inventors using two different types of 
sub-optimal decoders, including ordered IMMSE decoding and unordered zero-forcing 
decoding. In each simulation S was created using T=\ and independent 16-QAM 
modulation on each stream. Figure 2 is a graph comparing bit error rate performance 
10 between ordered IMMSE based sub-ML, ordered IMMSE, and ML decoding, using the 
decoding algorithm depicted in Figure 1 . With a 2-bit or 4-symbol vector search space, 
the decoder 100 displays a diversity order that is approximately the same as ordered 
IMMSE with about a 0.2dB gain. With a 3-bit or 8-vector search space, the BER curve 
for the decoder seems to have a diversity slope that lies between ordered IMMSE and ML 
15 decoding. At a BER of 10" 2 , the 8-vector search space decoder is approximately 1.2dB 
better than ordered IMMSE and 1 .2dB worse than ML. The 5-bit or 32-vector search 
space actually outperforms ML decoding up to a bit SNR of about 13dB. 

Figure 3 is a graph comparing bit error rate performance between unordered zero- 
20 forcing based sub-ML, ordered IMMSE, and ML decoding using the algorithm depicted 
in Figure 1. Unordered zero-forcing is much less complex than ordered IMMSE 
decoding. It can be seen that with a search space of 2-bits or 4-vectors, the decoder 100 
seems to perform exactly the same as ordered IMMSE. The diversity advantage of this 
advantage of this decoding scheme appears with a search space of 3-bit or 8-vector. The 
25 performance matches that of the ordered IMMSE based sub-ML decoding almost exactly 
here. At a BER of 10* 2 , the 8-vector search space decoder is once again approximately 
1 .2dB better than ordered IMMSE and 1 .2dB worse than ML. 

Figure 4 is a block diagram illustrating a two layer interference cancellation 
30 decoder 200 according to one embodiment of the present invention. Decoder 200 was 
found by the present inventors to also provide performance between sub-optimal and 
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MAP decoding with only a small complexity increase over sub-optimal decoding. In the 
case 7>1 this decoder operates column by column. A received signal vector y* is fed 
into a RSS decoder 202. Examples of the RSS decoder include, but are not limited to, 
sphere decoding, hierarchical decoding, concatenated RSS decoding, and the decoder 

5 proposed herein. RSS decoder 202 returns a hard symbol vector estimate of S . The 

A 

decoder 200 then decodes to S by performing interference cancellation via block 204 on 
each symbol, and then making a decision. This interference cancellation can be done 
many different ways using algorithms such as zero-forcing or minimum-mean- squared- 
error (MMSE) decoding. One example based on two-layer zero-forcing is set forth in 
10 steps 1-5 below, assuming that HU is the /nth column of H (estimated channel that may 
be estimated within another block or supplied externally). 

1. Setm = /. 

2. Compute Wm = (H™) + . 



15 3. Set Sk,m = Q 



f f Mi 



v 



4. Repeat steps 1-3 for m < (M + 1). 

5. Set the decoded symbol vector to s* = [$k,\ Ska ... J . 



Although decoder 200 is more complex when compared with a simple 
20 interference cancellation decoder, and does not perform as well as an RSS ML decoder, 
decoder has several advantages. One advantage of two layer interference cancellation 
decoder 200 is that it provides performance close to MAP decoding. Further, decoder 
200 is more easily implemented for systems using outer error control coding than are 
RSS decoders. Decoder 200 can be seen to have dramatically reduced complexity 
25 compared with MAP decoding techniques. The present inventors also found that decoder 
200 could be easily implemented as an additional stage to known hard decision RSS 
decoder implementations. 
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Decoder 200 was simulated by the present inventors using concatenated RSS ML 
decoding followed by zero-forcing. The systems were all simulated for a 2x2 antenna 
wireless system using T=l. Figure 5 is a graph comparing bit error rate performance of 
the decoder 200 shown in Figure 4 with ordered IMMSE and ML decoding when using 
5 T=l and 16-QAM on each stream and a 6-bit RSS ML decoder simulated for a flat fading 
channel. Decoder 200 can be seen to provide approximately a IdB performance gain 
over ordered IMMSE decoding. 

Figure 6 shows the bit error rate performance of an orthogonal frequency division 
multiplexing (OFDM) system in which decoder 200 employs a 3-bit RSS ML decoder 
simulated for a two path channel. The decoder 200 decodes the two element symbol 
vector sent on each frequency tone independently. The OFDM system forces each tone 
to have flat fading. It can be seen that decoder 200 provides approximately a 0.25dB 
performance increase over ordered IMMSE decoding. 

In summary explanation, a reduced search space minimum distance 
decoder/decoding algorithm has been described that provides average probability of error 
performance close to optimal MAP decoding. A sub-optimal minimum distance (MD) 
decoder employs the reduced search space minimum distance decoder/decoding 
algorithm to outperform known sub-optimal decoders used in multiple-input multiple- 
output (MIMO) communication systems. The decoder allows the system designer to 
make tradeoffs between complexity and performance by easily adjusting the search space 
size. As two layer interference cancellation decoder has also been described for MIMO 
communication systems, that also provides performance close to optimal MAP decoding. 
Although the two layer interference cancellation decoder is based on reduced subspace 
decoding, it is much easier to implement than corresponding reduced subspace ML 
decoders. 

In view of the above, it can be seen the present invention presents a significant 
30 advancement in the art of multiple-input multiple-output (MIMO) communication 
systems. Further, this invention has been described in considerable detail in order to 
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provide those skilled in the sub-optimal decoder art with the information needed to apply 
the novel principles and to construct and use such specialized components as are 
required. 

5 Further, in view of the foregoing descriptions, it should be apparent that the 

present invention represents a significant departure from the prior art in construction and 
operation. However, while particular embodiments of the present invention have been 
described herein in detail, it is to be understood that various alterations, modifications 
and substitutions can be made therein without departing in any way from the spirit and 

10 scope of the present invention, as defined in the claims which follow. For example, a 
sub-optimal decoder has been described which generates the decoded symbol estimate. 
Bit patterns E were chosen which were a subset of the total (Ns) A Mt possible patterns for 
further processing. The choice of these bit patterns could be 1) fixed once and for all 
(could be because of some pre-known knowledge); 2) fixed before-hand but changing 

15 from symbol to symbol; 3) determined once during reception and fixed for all symbols; 
and/or 4) changed from symbol to symbol. The choice of which E can be either arbitrary 
or based on some measure such as signal "strength", preprocessing SNR, probability of 
error, improvement in post processing SNR, total available computation resources, 
power, and the like, or any combination of the foregoing criterion. Those skilled in the 

20 art will appreciate that although E was chosen to correspond to the L weakest bits, and 
was predetermined and fixed, one could alternatively have used a threshold on the 
criterion and used only the bits which did not satisfy this threshold criterion. 

In the choice where E was chosen to correspond to those bits from some L bit 
25 locations, the L bit locations could be 1) fixed before hand for all symbols; 2) fixed 
before-hand but changing from symbol to symbol; 3) adaptively determined from some 
training or the received signal and fixed for all symbols; and/or 4) adaptively changing 
from symbol to symbol. Again, the choice is arbitrary or based on some measure such as 
signal "strength", preprocessing SNR, probability of error, improvement in post 
30 processing SNR, total available computation resources, power, and the like, or some 
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combination of the foregoing criterion. Although a fixed L was described herein above, 
the locations were determined by a "weakest criterion" for all symbols. 
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